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ABSTRACT 



How students used computers and Internet access in the 
day-to-day work of the problem-based learning (PBL) classroom was studied 
with second year medical students working in small learning groups. Fifteen 
students, one from each PBL group, met as a focus group to discuss computer 
and Internet use. Several themes emerged from the discussions. Overall, there 
was a generally positive effect of the use of network technology on the PBL 
discussion process. The role of the preceptor appeared crucial for the group 
sessions and for the use of electronic resources. The technology was used for 
several different reasons, especially "foraging" for information or targeted 
hunting for specific information. The use of the technology appeared to 
influence students’ experience of medical education in terms of electronic 
cooperation. (SLD) 
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Introduction 

Our institution uses problem-based learning (Barrows, 1992) in its second year basic 
science curriculum. 1 Traditional discipline-based content is divided into 10 integrated "domains," 
e.g., "Neuromuscular," "Disorders of Development and Behavior," "Infectious 
Disease/Immunology.". Each domain is governed by a "curriculum development group" 
responsible for determining domain content, PBL case materials, a "preceptor guide-book," and 
final examination questions. Each domain is from 2 to 4 weeks in length. The creation of new 
classrooms connected to the Internet now allows students to access information electronically 
during the course of PBL case discussions. We attempted to describe how a modest 
incorporation of technology is being adopted by 2 nd -year medical students in their PBL small 
groups. 

Rationale 

Numerous calls for developing the information management expertise of medical 
students have spurred information technology curricular initiatives country-wide. Examples 
include the University of Rochester, Columbia University, and the University of Wisconsin. 2,3,4 
The Association of American Medical Schools, in its Medical School Objectives Project, Report 
Two (1998), specifies roles in the domain of medical informatics that medical school graduates 
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should be prepared to fulfill. Of particular relevance is the role of life-long learner." 5 According 
to the report: 

"Support of life-long learning with information technology requires more than 
computer literacy. Other requirements include cognizance of the broad range of 
medical information resources available and their relative value for particular 
needs, the know-how to use them, and the motivation to use them routinely." 

(page 6) 

The PBL classroom, with its reliance on student-determined learning agendas, is fertile 
ground for implementing the kinds of information-seeking skills proposed in the AAMC report. 

Koschmann (1995) describes three ways to incorporate the use of technology into the 
medical school curriculum that he calls "learning about, learning through, and learning with" 
computers. "Learning about computers" refers to traditional computer literacy and competence. 6 
"Learning through computers" refers to using computers for instructional delivery, e.g., computer 
assisted instruction, web-based courses, etc. "Learning with computers" is requiring students to 
use computers in their work on a day-to-day basis. It is this latter form of learning that 
Koschmann believes to be the most powerful means of fostering life-long learning in medical 
students. 

Our research focused on the use of computers in the day-to-day work of the PBL 
classroom. It was our hope that the research would shed light on what "learning with computers" 
by 2 nd year medical students looks like in action, and offer suggestion so other programs 
contemplating similar initiatives. 
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Researchers have investigated the integration of technology into the problem-based 
learning (PBL) small group discussion in a variety of ways. Koschmann, et all (1995) described 
the development of a "technology-enriched facility" - the Collaborative Learning Lab- which 
incorporated a number of software tools to support collaborative learning in medical education. 7 
Jost, et al (1997), in a graduate education class, supported tutorial sessions using HyperCard 
stacks to record student information and views on learning and technology; software tools to help 
facilitate the tutorial sessions; and learner self-evaluations sent via email. 8 Ram (1999) described 
a number of ways technology was used to aid the PBL process, including web sites designed to 
provide information specific to a problem and online discussions to augment classroom 
discussions. 9 Each researcher described a unique educational research context. Koschmann's 
work was undertaken in a very technology rich environment in which each student was supplied 
a networked computer and software tools developed specifically to support the PBL process. 
Jost's investigation focused primarily on comparing a project-based versus a PBL-based 
approach to collaborative learning. While the use of technology was included in the study, it was 
only one of several variables of interest. Ram was interested in the power of PBL to teach 
problem solving skills to undergraduates while keeping them motivated to leam, rather than 
focusing primarily on the use of technology to support the PBL process. Our research sought to 
contribute to our knowledge of how technology is implemented into the PBL classroom in a 
medical education context. Rather than focusing on a technologically rich environment as did 
Koschmann, our resources were considerably more modest: one networked computer connected 
to a projector was installed in each of our new PBL small group discussion rooms. In addition, 
there were data ports installed on the tables, six to a table. 
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Research Questions 



The study examined how do students use network/Internet access during PBL small 
group. The specific research questions were: 

What did the students use the new technology for? 

What URLs did students access? Which were useful? 

Were there any problems students encountered trying to use the technology? 

How did students respond to using the technology in the PBL small group? 

How did students' use of the technology change over the course of the year? 

Which students used it? Were there any who didn't, and why? 

Did the use of tech in small groups add to or distract from the group's focus on the case? 
Method: Implementation of the Technology 

The implementation of the technology include both faculty development and student 
orientation components. The faculty development component consisted of a series of domain- 
based workshops focusing on possible uses of the new technology in small groups, plus domain- 
specific lists of web-based resources. Student orientation included a handout describing efficient 
information searching strategies and a handout describing relevant web-based resources for a 
given domain. 
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Methods: Research 



Scheduling of PBL groups at Michigan State University's College of Human Medicine is 
unique in that group membership is "shuffled" twice at equal intervals throughout the course. 
Thus, rather than using (a) PBL group(s) as a means of identifying study participants, one 
participant from each initial PBL group was asked to participate in the study. This allowed the 
researchers to sample a variety of opinion from the widest possible number of PBL groups. 

A sampling of 15 students, one from each PBL small group, was initially created to serve 
as a focus group for the research. We sought to balance participation according to gender and 
minority representation. Approximately 50% male/female, 20% minority, were invited, both of 
which represent the prevalence of these groups in the College. The first meetings were scheduled 
for September 12 and 28, 2000. Two more meetings were arranged for January 29 and 30, 2001, 
and two more for Marchl9 and 20, 2001. The research questions presented above served as the 
framework for the focus group discussions. The focus groups discussions were recorded on 
written transcripts, and the transcripts were analyzed for themes that addressed the research 
questions. 

As a result of comments made by participants during the September focus groups, we 
added 3 questions to be asked of the January groups. 

Results 



The study results are presented within the framework of the questions presented to the 
focus groups. Notes were taken at the meeting by the second author, and are reproduced without 
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editing as Appendix 1. In order to distinguish results obtained from the 2 focus groups conducted 
during each month, the results of the first group (e.g., September 12) are in black, while those of 
the second group (e.g., September 28) are in italics. Questions 1 through 10 were asked of both 
the September and the January groups. Questions 11 through 13 were asked only of the January 
groups. 

Of the 15 students who originally agreed to participate in focus groups, 10 participated 
during the September groups, 7 in January, and 4 during the March group. Results for the March 
focus group were unavailable for incorporation into this report. Absent participants gave as their 
reasons "participating in Q-ball" (study) groups" or "forgot." 

It should be noted that the results obtained for Question 9: "In general, how did students 
respond to using the technology in the PBL small grou?" were somewhat contaminated by 
participants "splitting their vote" into two categories. For example, 5 participants during the first 
focus group (September 12) noted that they responded "somewhat negatively" to being unable to 
use paper cases, but 4 of the 5 responded "somewhat positively" to all other aspects of using 
technology in the PBL small group. 

The March 19 meeting did not take place due to lack of student participants. The 
investigators are contacting the participants to determine reasons for non-attendance. 

Discussion 

It was the purpose of the research to shed light on how implementation of a relatively 
modest technological innovation affected the PBL small group process. This discussion is in the 
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form of thematic descriptions of how the implementation of the network technology affected the 
PBL small group case process, supported by data and quotes from the focus group transcripts. 

Theme 1: Overall, there was a generally positive effect of the use of network technology on the 
PBL discussion process. 

When asked to rate how students responded to using the technology in the small group, 

14 out of 21 selected "somewhat positively" or "very positively.” None selected "very 
negatively", and 6 selected "somewhat negatively," 5 of which were based upon a desire for 
written case materials. It should also be noted that, in response to Question 7: Did the use of the 
technology in the small group add to or distract from the group's focus on the case, the positive 
comments appeared to outweigh the negative comments slightly. There were 9 negative 
comments, versus 13 positive comments and one neutral comment. 

Example positive comments : 

"Most of the time it added, nice when we had a vague idea of what to know, and the 
technology would help us to refresh on the spot." 

"Added: instead of discussing "why" with respect to hypotheses, we could look it up." 
"Definitely a positive addition to the rooms/curriculum, despite problems. Overall, 
valuable to learning." 

"Overall, facilitates understanding of things I'd be too lazy to look up on my own. Would 
need a library of books otherwise. We seem to do more "looking up" in class, when in the 
past we would say we'd do it after class, and then forget about it." 
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Example negative comments : 

"(We were) distracted because the wait for result caused an increase in side 
conversations. 

"We were distractible, so at times it would help us get off track, e.g., Superbowl 
commercials." 

"Frustrating when images not "tagged" - no way to identify salient features in an x-ray." 



Theme 2:The role of the preceptor appeared to be crucial. 

While we did not specifically ask questions about the role of the preceptor during the 
September focus groups, participants then voiced opinions about the effect of the preceptor on 
the use of technology. We included a question specifically addressing the role of the preceptor 
(Question 13) for the January focus groups. 

There appeared to be a "preceptor effect." A number of comments referred to the value 
students placed on a preceptor who encouraged use of and modeled electronic resources. Other 
students remarked on the lack of use of the technology as being tied to a lack of preceptor 
modeling. Still another student noted that their preceptor could "go overboard" on encouraging 
student technology use! 

Example comments: 

"Works if preceptor suggests a search or offers a site." 
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"Previous preceptor encouraged use, now and in the future. Now, Dr. X knows 
everything (so no need to go online)." 

" 1 st time preceptor - tried to get everyone to use it, students got tired of it, then he let up, 
which helped group dynamics. He also "brought up" web sites, too." 

"Neither promoted nor hindered, a lot (of preceptors) are not comfortable with it, they 
leave it to us." 

"(Preceptor) can do more during the group." 

"Inexperienced preceptor would/might not encourage use." 



Theme 3: The technology appeared to be used for several different reasons. 

There were a variety of uses to which the technology was put. There were examples of 
"foraging" for information by entering a learning issue into the search field of a web search 
engine, most often ":Google.com", to see what would come back. There were also examples of 
targeted hunting for specific information, for example, obtaining images of diseases not available 
on the course web site. 

Example comments: 

"Look up lab tests, etc that were not in books" 

"Make charts, graphs in Excel" 

"Look up x-rays" 

"Look up symptoms of diseases" 

"Look up treatments" 

"Searching for simple stuff like normal lab values" 
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"Finding relevant images, x rays" 

"Finding incidence rates" 

"Things on content list, inadequately covered in lecture or texts, to get more info, e.g., 
"hypoplastic left heart"" 

"Couple of searches conducted 1 st week (of a domain), recall of previously learned 
content " 



Theme 4: The use of the technology appears to be influencing students' experience of medical 
education in terms of "electronic cooperation. " 

There appeared to be the formation of an "electronic community" built around the sharing 
of useful web sites by posting URLs to the students electronic discussion group. Sites such as the 
Utah Medical School's pathology images would be sited by a number of participants, having 
learned about them from their colleagues in other PBL groups. This sharing of electronic 
resources bespeaks a welcome phenomenon of "electronic community" among students in the 
PBL course. 

Example comments: 

"Someone emailed all learning issues to people in the group" 

"Utah - found something specific outside, then brought it in" 

"Discussion of the audio/visual stuff stimulated good discussions." 

"Bring notes to class on laptop." 
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The 4 themes presented above indicate that the integration of network technology in the 
PBL class discussion, even on a small scale of one computer and Internet accessibility, provides 
significant "value added" to the discussions in terms of access to additional resources and 
facilitating the discussion process, particularly with respect to review of previously learned 
content, and provision of additional clarifying information. Preceptors need to keep a steady 
hand on the wheel, however, to keep the focus on the case and on relevant use of the computer in 
the face of possible distraction. 

There are several limitations to this study. Attendance at the focus groups was very 
inconsistent; indeed, one meeting had to be canceled for lack of participants. The most common 
excuse was a conflict with early evening study groups. This variability in attendance probably 
decreased the reliability of data collected by decreasing sample size. 

The study relied on student self-reports presented in the presence of the course 
coordinator. It should be noted that while the presence of such an individual may bias student 
comments toward the more favorable, it did not appear to have such an effect, given participants 
willingness to share negative comments. 

In spite of these limitations, we found the study valuable, both as an internal formative 
exercise and as a source of lessons to others. We decided to make the paper cases available to 
students, based upon feedback obtained during the September focus groups, that students missed 
the ability to take notes on the case materials. We now have an increasing repository of useful 
sites for individual domains, again based upon students' feedback. We also have a better idea of 
when to target student technology training, such that it meets a perceived need during PBL case 
discussions. 
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Other institutions may take away several lessons. First is the need for faculty 
development, both to address the role of technology in the PBL case analysis process, and to 
address possible faculty skill deficits with respect to technology. Second is the need to alert 
preceptors to keep a steady hand on the use of technology: those groups that are predisposed to 
wander in their attention will have even greater opportunity for going off on tangents, or even 
abandoning their focus on the case analysis process entirely, with the introduction of easily 
available Internet access during small groups discussion. Last, those institutions that are 
considering implementing Internet access during PBL discussion need not fear greatly that it will 
disrupt such discussion unduly, but rather that such technology will serve both as an additional 
resource, and as a target of increased information management skill development, in the context 
of the PBL classroom 
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